Photography is the process of making pictures by means of the action of light. Light is the commonly used term for electromagnetic radiation in a frequency range that is visible to the human eye. Light patterns reflected or emitted from objects are recorded by an image sensor through a timed exposure. Image sensors can be chemical in nature, such as photographic film, or solid state in nature, such as the CCD and CMOS image sensors employed by digital still and video cameras.
Digital cameras have a series of lenses that focus light to create an image of a scene. But instead of focusing this light onto a piece of photographic film, as in traditional cameras, it focuses it onto the solid state image sensor which converts the electromagnetic radiation of the light into an electrical charge. The atomic element of an image sensor is said to be a picture element, or a ‘pixel’ and practical image sensors for digital photography typically have a large number of pixels. The electrical charge indicates a relative intensity of the electromagnetic radiation as perceived by the image sensor, and generally is used to associate a light intensity value with the pixel.
One goal of photography is to provide an image that accurately represents the image viewed by the human eye. However, the human eye is not equally sensitive to all wavelengths of light. As a result, the response of the image sensors to the electromagnetic radiation that impinges upon them must be adjusted in accordance with the sensitivities of human vision. The adjustment is typically done by the adjusting the exposure of the image sensor to the electromagnetic radiation to compensate for human sensitivities to different wavelengths.
However it is often difficult to determine the correct level of exposure to provide to the image sensor; the failure to determine and apply the correct level of exposure of electromagnetic radiation to the light sensitive image sensor component results in degradation of the captured image. Such degradation is often referred to as ‘overexposed’ or ‘underexposed.’ Overexposure occurs when the level of electromagnetic radiation that is exposed to the light sensitive component is greater than the optimal level for the light wavelength. Overexposure often results in lack of highlight detail in the captured image.
FIG. 1A illustrates the response characteristic of an image sensor to overexposure. The abscissa represents the intensity of the electromagnetic radiation impinging upon the sensor. The ordinate represents the corresponding output voltage of the image sensor. For low intensity radiation, the voltage of the image sensitive component increases linearly with the intensity of the electromagnetic radiation. However, as the intensity of the electromagnetic radiation increases beyond a threshold To (as indicated by the dashed line of FIG. 1a), the image sensitive component output voltage does not change accordingly, but rather levels out at some maximum voltage. Thus, the image sensitive device cannot accurately represent images having electromagnetic radiation intensity levels that exceed the threshold To and the recorded image is said to be overexposed.
FIG. 1B illustrates the response characteristic of an image sensor to underexposure. Underexposure occurs when the level of exposure of the image sensitive component to the electromagnetic radiation is less than the optimal level. Underexposure often results in lack of shadow detail in the captured image. As can be seen in FIG. 1B, for low intensity values the voltage of the image sensitive component does not change in response to changes in the intensity of the electromagnetic radiation. Only after the light intensity is past a minimum light intensity level threshold Tu does the voltage of the image sensitive component increase linearly with the intensity of the electromagnetic radiation. Thus, the image sensitive device cannot accurately represent image details or colors having electromagnetic radiation intensity levels that do not exceed the threshold TU and the recorded image is said to be underexposed.
FIG. 1C illustrates a transfer function that reflects the correct exposure of an image sensor to electromagnetic radiation. Correct exposure occurs when an image is captured optimally with full detail in the segments of low electromagnetic radiation intensity levels as well as in the segments of high electromagnetic radiation intensity levels. Correct exposure allows both highlights and shadows sections of the original image to be accurately represented in the captured electronic representation of the image.
FIGS. 2A-2C illustrates histograms of pixel intensities of exposed images, wherein the exposed images each have a range of potential pixel intensities from 0 through 255. FIG. 2A is a histogram of pixel intensities that may be found in an image that has been overexposed, for example using an exposure process having a transfer function such as that illustrated in 1A. FIG. 2B is a histogram of pixel intensities that may be found in an underexposed image, such as an image captured using the transfer function of FIG. 1B. FIG. 2C is a histogram of pixel intensities that may be found in an image that has been correctly exposed, for example using the transfer function of FIG. 1C. While the histogram of FIG. 2C illustrates a normalized distribution of pixel intensities, FIG. 2A illustrates that an overexposed image has pixel intensities that are compressed at the maximum image sensor output value (‘255’), while an underexposed image has pixel intensities that are compressed at the minimum image sensor output value (‘0’).
An image reaching the pixel array contains dark and light areas that can differ drastically in intensity and thus cause the image to have high dynamic range. Although the dynamic range of the image might fit within the dynamic range of the pixel array the exposure calculation must be extremely accurate. The limited dynamic range of realistic pixel arrays and the high dynamic range of typical images combine to make accurate exposure calculation a very difficult task.
The current art recognizes a number of methods of calculating the exposure time. One known method is to capture the image using an estimated exposure time setting, observe the pixel output histogram, arrive at a new exposure time estimate and acquire the image using this new exposure time estimate. This method is wasteful of power as it captures each image twice. It is also inaccurate as the second exposure time estimate, although better than the first value, is still an approximation and numerous pixels will be exposed to light intensities that are outside of their dynamic range.
Another method is to examine the entire image, measure the average light image intensity value and calculate an exposure time based on this value. However, such a method is inaccurate as it does not take into account the actual distribution of light intensities within the image. Here too numerous pixels will be exposed to light intensities that are outside of their dynamic range.
Identifying the correct amount of exposure is further complicated by photometry. Photometry is the science of measurement of visible light, especially its intensity and it can be used to describe the image intensity in terms of its perceived brightness to human vision. Photometry can be used to account for the different sensitivities of human vision to light wavelengths by weighting the measured intensity of a wavelength with a factor that is a function of how sensitive the eye is at that wavelength.
Photometric measurements are typically made and reported using electromagnetic radiation sensitive devices that are designed for the specific photometric measurement function. The electromagnetic sensitive device may be, for example, a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) device. FIG. 3 illustrates a pixel array 10 which includes pixel structures such as pixel structure 12 and photometric measurement structures such as photometric measurement structure 14. Typical pixel structures using such photometric measurements devices take photometric measurements in one or more sub-areas of the image to be captured. The measurements of the various sub-images are then processed to determine a level which the image sensitive material should be exposed to electromagnetic radiation. The image undergoes a second processing stage where the image is then captured by exposing the solid-state image sensor device, to the incoming electromagnetic radiation for the pre-calculated exposure time.
However, because the photometric measurements are taken from a generally small subset of points within the image to be captured it is often sub-optimal; any visual attributes and/or artifacts of the sampling points are used to calculate an exposure level that is applied across the image. As a result, exposure times derived from such measurements are often inaccurate and the resulting processed image generally suffers. Another disadvantage of the described method is that it does not consider properties of the image sensor. For example, image sensors may react differently to different intensities of electromagnetic radiation. Blindly applying a weighting factor to the image sensors that does not take into account the gain characteristics of the image sensors may exacerbate the problem of providing an optimal output image.